Sparse Nonlinear Regression: Parameter Estimation under Nonconvexity
نویسندگان
چکیده
We study parameter estimation for sparse nonlinear regression. More specifically, we assume the data are given by y = f(x�β∗) + �, where f is nonlinear. To recover β∗, we propose an �1regularized least-squares estimator. Unlike classical linear regression, the corresponding optimization problem is nonconvex because of the nonlinearity of f . In spite of the nonconvexity, we prove that under mild conditions, every stationary point of the objective enjoys an optimal statistical rate of convergence. Detailed numerical results are provided to back up our theory.
منابع مشابه
Sparse Nonlinear Regression: Parameter Estimation and Asymptotic Inference
We study parameter estimation and asymptotic inference for sparse nonlinear regression. More specifically, we assume the data are given by y = f(x>β∗) + , where f is nonlinear. To recover β∗, we propose an `1-regularized least-squares estimator. Unlike classical linear regression, the corresponding optimization problem is nonconvex because of the nonlinearity of f . In spite of the nonconvexity...
متن کاملNonconvex Sparse Logistic Regression with Weakly Convex Regularization
In this work we propose to fit a sparse logistic regression model by a weakly convex regularized nonconvex optimization problem. The idea is based on the finding that a weakly convex function as an approximation of the `0 pseudo norm is able to better induce sparsity than the commonly used `1 norm. For a class of weakly convex sparsity inducing functions, we prove the nonconvexity of the corres...
متن کاملOnline State Space Model Parameter Estimation in Synchronous Machines
The purpose of this paper is to present a new approach based on the Least Squares Error method for estimating the unknown parameters of the nonlinear 3rd order synchronous generator model. The proposed method uses the mathematical relationships between the machine parameters and on-line input/output measurements to estimate the parameters of the nonlinear state space model. The field voltage is...
متن کاملVariable Selection for Sparse High-Dimensional Nonlinear Regression Models by Combining Nonnegative Garrote and Sure Independence Screening.
In many regression problems, the relations between the covariates and the response may be nonlinear. Motivated by the application of reconstructing a gene regulatory network, we consider a sparse high-dimensional additive model with the additive components being some known nonlinear functions with unknown parameters. To identify the subset of important covariates, we propose a new method for si...
متن کاملOptimal Linear Estimation under Unknown Nonlinear Transform
Linear regression studies the problem of estimating a model parameter β* ∈ℝ p , from n observations [Formula: see text] from linear model yi = 〈xi , β*〉 + ε i . We consider a significant generalization in which the relationship between 〈xi , β*〉 and yi is noisy, quantized to a single bit, potentially nonlinear, noninvertible, as well as unknown. This model is known as the single-index model in ...
متن کامل